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(54) A video-data encoder and recording media wherein a video encode program is recorded 



(57) To provide an apparatus for embedding water- 
mark data according to tfie frequency-doniain method 
efficiently Into video data, a video-data encoder of the 
invention for performing encoding of an original video 
data into a compressed data stream having a mutti-layer 
structure comprises; means (101) for transforming the 
original video data Into a sequence of processing units 
of frequency domain data; means (103) for embedding 
predetermined watermark data into at least one unit of 
the sequence of processing units; and means (104 and 



105) for generating the compressed data stream by 
processing the sequence of processing units. There- 
fore, the data transformation of the original video data 
Into the frequency domain data for embedding the 
watermark data can be performed efficiently without any 
additional process, by exploiting tfie data transformation 
for encoding the original video data into the compressed 
data stream. 
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1 EP0 860 

Descrfption 

The present invention relates to modification of 
video data, and particularly to that for synthesizing iden- 
tification data thereto for identifying the video data. s 

Illegal reproduction of digital video data is an actual 
problem. For preventing the illegal reproduction, there is 
proposed a playt>ack system wherein digital video data 
are encoded into cipher data which can t>e played back 
only by video players equipped with a proper decipher w 
key. However, even the ciphered video data, the illegal 
reproduction canrxit be preverrted once they are deci- 
phered. 

Hence, embedding special information into the dig- 
ital video data themselves is attempted for preventing is 
the illegal usage and reproduction for preverrting the ille- 
gal usage and reproduction. In the following para- 
graphs, data representing this special information will 
be called the electronic watermark data. 

The electronic watermark data can be classified 20 
into two kinds, that is, visft)le electronic watermark data 
and invisible electronic watermark data. 

An example of the visible electronic watermark data 
is disclosed in a Japanese patent application laid open 
as a Provisional Publication No. 241403/%. In the 25 
exarrpie, for synthesizing the electronic watermark data 
into original picture data, brigfitness values of original 
pixel data corresponding to opaque pixel of the elec- 
tronic watermark are modified leaving color differential 
components thereof unchanged. When modifying the 30 
brightness values, scaling factor to be applied for the 
modification may be determined according to color 
conponent, random nurTt>er. pixel value of the elec- 
tronic water mark, or others. 

Thus, the visible electronic watermark modifies a 35 
picture so that the rrxxfif ication may be sensed visually 
by corrparing. or even without comparing, to its original. 
by synthesizing special characters or marks into the pic- 
ture. H^Ke, the visible electronic watermark is effective 
to appeal the prevention of illegal use to the observer. 40 
However, degradation of picture quality is, more or less, 
unavoidabia 

On the other hand, the invisible electronic water- 
mark is the watermark unable to be sensed visually, 
t>eing embedded within image data not to degrade the 4S 
picture quality. 

By embedding, into a video program, special infor- 
mation for identifying its licensee, for example, as the 
invisible electronic watermark, identification of the licen- 
see becomes possit)le by extracting the watermark data so 
from illegal copies of the video program. It becomes 
also possit)le. by embedding reproduction-prohibitive 
information in the program, to alarm or restrict a user to 
make duplication by a VTR (Video Tape Recorder), by 
functioning a copy prevention mechanism therein, for ss 
exarrple, when the reproduction-prohibitive information 
is d^ected by the VTR. 

The invisible electronic watermark data may k>e 
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embedded into a part of image data which gives little 
affect to the pk;ture quality, such as into LSD (Least Sig- 
nificant Bit) of each pixel value. 

However, when the watermark is embedded into 
the LSB, it can be easily eliminated by way of a low- 
pass fitter. Further, the image data compression gener- 
ally stands on reducing data amount by omitting data 
parts giving littie affect to the picture quality. That means 
the watermark embedded there is also eliminated by tiie 
image data corrpression. 

As above descrbed. tiiere has been a tradeoff 
between the picture quality arxJ the traceability of tiie 
watermark in tiie invisible electronic watermark. 

For evading this tradeoff, a method of embedding 
tiie watermark data into frequency spectrum data of an 
image is proposed (in p. 13 of the NIKKEI 
ELECTRONICS, no. 660. 4/22/1996). In this method, 
which will be called the frequency-domain method in tiie 
following paragraphs, being embedded into frequency 
components, tiie watermark data have sufficient dura- 
bility to the image data processing such as data com- 
pression or filtering, and further, interference between 
different watennark data is also prevented, making diffi- 
cult to break the watermark data wrtiiout giving serious 
affects throughout the picture. 

In the frequency-domain method, tiie electronic 
watermark is emt>edded as folkTws, for exarrple. 

Original image data are transferred into frequency 
componerrts by way of the DOT (Discrete Cosine Trans- 
form). Selecting n components /(I), f(2), .... f{n) each 
having higher n values among the frequency compo- 
nents thus obtained, an electronic watermark data set 
iv(1), w{2) w{n) is prepared so as to accord to a nor- 
mal distribution having a mean value 0 and a variance 1 . 
Then, F(l) = f{i)+a\f{i)\xw{t) is calculated for each / 
(/ = 1, 2. .... r?). a being a scale factor. From the fre- 
quency components wherein each f{i) is replaced with 
F(i), tiie image data including tiie electronic watennark 
are obtained. 

For detecting the electronic watermark, following 
processes are performed, for exarrple, on condition that 
the original image data and the probable watermark 
data set (/ = 1 . 2 n) are known. 

Rrst. frequency components F(1) to F(n) corre- 
sponding to those f{\) to f{n) of the original image as 
atx>ve defined are extracted from image data wherein 
the electronic watermark is considered to be embed- 
ded. Then, each /-th conponent W(i) of a prot>able data 

set vector IV = (IV(1), IV(2) W{n)) is calculated as 

Mi) = (^(O ■ Wiyfli) ■ When a static resemblance C 
of the probable data set vector IV to the watermark data 
set vector w = (iv(1). w(2), .... w[n)) obtained from their 
normalized inner product represented by 

C =W*w/(\W\*\w\) 
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is larger than a certain value, the concerning electronic 
watermark is deternnined to be ennbedded in the object 
image data. 

Thus, the author of the original image can effec- 
tively verify illegal copies by detecting the electronic s 
watermark emt>edded by the frequency-domain method 
in the digital image data according to each licensee. 

FIG. 6 is a block diagram illustrating a data nrxxlifier 
to be applied to the prior art for emtjedding the elec- 
tronic watermark data by way of the frequency-domain io 
method. Original image data 601 are converted Into fre- 
quency components by a DOT processor 602. which 
are modified by a watermark enft>edder 603 according 
to a watermark data set 604 and converted by an 
inverse DOT processor 605 into time-domain data to be is 
output as the watermarked image data 606. 

Suppose here to apply the atx>ve frequency- 
domain method to MPEG (Moving Picture Expert 
Group) vkteo data. FIG. 7 is a block diagram illustrating 
t>ask: conf iguratbn of a video-data encoder 700 for gen- 20 
erating the MPEG video data, wherein are comprised a 
DOT processor 101 for converting an original video data 
stream 110 into frequerKy-domain data, a quantizer 
104 for thinning out negligible data therefrom and a var- 
iable-length coder 105 for encoding the quantized data 25 
into the MPEG data stream 120. As can be seen from 
FIGs. 6 arxJ 7. two times of DOT should k>e performed 
when the data nrxxirf ier of FIG. 6 is directly applied in the 
video-data encoder 700 of FIG. 7. The erKXXling proce- 
dure of the video data into compressed data charges a 30 
heavy load to the encoder needing a large anrxHjnt of 
operation. Therefore, operational lo^ for embedding 
the electronic watermark is expected, without saying, to 
be as small as possO^le. 

Th^efbre, a primary object of the present invention 35 
is to provkJe a method of and an apparatus for embed- 
ding watermark data according to the frequency- 
domain method efficiently into vkieo data. 

In order to achieve the object a vkieo^lata encoder 
of the invention for perfomning encoding of an original 40 
video data into a compressed data stream having a 
multi-layer structure comprises; 

means for transfornrting the original video data into 
a sequence of processing units of frequency 45 
domain data, said processing units being compo- 
nents of a predetemnined layer of the multi-layer 
structure; 

nrteans for embedding predetermined watermark 
data into at least one unit of the sequence of so 
processing units; and 

means for generating the compressed data stream 
by processing the sequence of processing units. 

Therefore, the data transformation of the original ss 
video data into the frequency domain data for embed- 
ding the watermark data accorcfing to the frequency- 
domain method can t>e performed eff identty without any 



additional process, exploiting the data transformation 
for encoding the original video data into the compressed 
data stream. 

Further,-in an embodiment of the invention, where 
the encodng is performed according to the MPEG 
standard, the watermark data are embedded into one or 
some of macro-blocks determined referring to a crite- 
rion defined in connection with anyone of a slice, a pic- 
ture, a f iekl, a frame, or a GOP. 

Therefore, ^itional bad of the video-data 
encoder for embedding the watermark data can be still 
suppressed. 

The foregoing, further objects, features, and advan- 
tages of this invention will t>ecome apparent from a con- 
sideration of the following description, the appended 
claims, and the accorrpanying drawings wherein tiie 
same numerals indicate the same or the corresponding 
parts. 

In the drawings: 

FIG. 1 is a block diagram illustrating a first embodi- 
ment of the invention; 

FIG. 2 is a flowcfiart illustrating operational flow of 

the video-data encoder 100 of FIG. 1 ; 

FIG. 3 shows a part of layer structure of tiie MPEG 

data; 

FIG. 4 is a block diagram illustrating configuration of 
a videoKJata encoder 400 according to a second 
emtxxJiment of the invention; 
FIG. 5 is a f bwchart illustrating operational flow of 
the video<lata encoder 400; 
FIG. 6 is a block diagram illustrating a data nxxjifier 
to be applied to a prfor art for embedding the elec- 
tronic watemiark by way of the frequency<fomain 
metfiod; and 

FIG. 7 is a bkxk diagram illustrating basic corrfigu- 
ration of a video-data encoder 700 for generating 
ttie MPEG data. 

New. emtxxiiments of the present inventfon will t>e 
described in connection with the drawings. 

FIG. 1 is a block diagram illustrating a first emtxxli- 
ment of the invention, wherein a video-data encoder 
100 according to the ennbodiment comprises; 

a DOT processor 101 for performing Discrete 
Cosine Transform of an original image data stream 
110 wherein electronic watermark data 102 are to 
be embedded, 

a watermark emt)edder 103 for modifying output of 
ttie DCT processor 101 according to the electronic 
watermark data 102, 

a quantizer 104 for thinning out negligible data from 
output of the watermaric embedder 1 03, and 
a variat>le-length coder 105 for generating variable- 
length code data from output of the quantizer 104 to 
k>e output as an MPEG data stream 120. 
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FIG. 2 is a fbwchart illustrating operational flow of 
the videchdata encoder 100. 

Data of the original image data stream 11 0 are read 
out (at step SI) block l>y block (each having 8 x 8 pixels) 
arxl transformed into frequency-domain data by the 5 
DOT processor 101 (at step S2). Each time when data 
corresponding to a macro-block (having 16x16 pixels) 
are transformed, the electronic watermark data 1 02 are 
embedded into the macro-btock (at step S3) by the 
watermark embedder 103 as described pre/busty con- io 
cerning the frequency-domain method. The macro- 
block modified with the electronic watermark data 102 is 
quantized t>y the quantizer 104 (at step 84), and coded 
into variable-length codes (at step S5) by the variable- 
length coder 1 05 to be output (at step SB) as the MPEG is 
data stream 120. 

Thus, the electronic watermark data 102 can be 
embedded into a video data without needing any addi- 
tional DCT processing in the emtxxJiment, exploiting the 
DOT process performed along with the MPEG encodng 20 
of the video data. Here, each of n components of data 
set IV representing the electronic watermark data 1 02 is 
embedded into each of n largest frequency components 
of the frequency-domain data. Therefore, the electronic 
watermark data 102 are least affected by the quanttza- 25 
tion performed ty the quantizer 104. 

In the following paragraphs, a second embodiment 
of the inverrtion will be desaibed refem'ng to FIGs. 3 to 
5. 

FIG. 3 shows a part of layer structure of the MPEG 30 
data. A GOP (Group Of Pictures) layer 303 comprises 
picture layers I (Intra-picture), P (Predictivei>icture) and 
B (Bidirectionally Predctive-picture). Each (302) of the 
picture layers comprises slice layers, each (301) con- 
sisting of the macro-t)locte denoted by M. As for the pk;- 35 
tures, fields or frames may t>e assigned. 

In the first embodiment, fhe electronic watermark 
data 102 are embedded for every macro-block along 
with the MPEG eiKoding procedure. However, total 
operatbnal load still arrxHjnts considerably high since 40 
the MPEG encoding itself needs a large amount of 
operatfon. Therefore, the ^ectronic watermark data 102 
are erTi>edded into one or some of macro-t)locks of 
every slice layer in the second embodment. for lighten- 
ing the cperatkxial foad of the MPEG encoding proce- 45 
dure including embedding the electronic watermark 
data 102. 

FIG. 4 is a block diagram illustrating configuration of 
a videodata encoder 400 according to the second 
embodiment, wherein a maao-block selector 40 1 is fur- so 
ther comprised in addition to the vkleo-data encoder 
100 of FIG. 1. Operation of ttie video-encoder 400 is 
illustrated by a f kswchart of FIG. 5. 

At steps SI and 82, the original image data stream 
110 is transformed bkx:k by block into frequency- ss 
domain data, in the same way in the first embocfinrienl 
When the watermark embedder 103 receives a macro- 
block, the macro-block selector 401 incficates whether 



the watermark embedder 103 should or not embed the 
electronic watermark data 102 into concerning macro- 
bkx;k (at step T1). referring to a criterion predetermined 
for a slice layer, in the second embodiment Into the 
macro-t>locks indicated to embed the electronic water- 
mark data 102. the electronic watermark data 102 are 
embedded (at step S3) by the watermari^ embedder 
103, other macro-blocks being output as they are. The 
output of the watermark embedder 103 is quarrtized (at 
step 84) t>y the quantizer 104, and coded into variable- 
length codes (at step 85) by the variat)l6-length coder 
105 in the same way with the first embodiment for gen- 
erating the MPEG data stream 120 (at step 86). 

Thus, the operational load of tiie MPEG encoding 
procedure including embedding the electronic water- 
mark data 102 can be lightened still more than the first 
embodiment by limiting number of macro-blocks in a 
slice layer wherein tiie electronic watermark data 102 
are embedded, in the second embodiment 

Heretofore, the present invention is descn'bed in 
connection with the above two emtxxfiment. However, it 
can be easily understood that various applications can 
be considered in the scope of the present invention. For 
example, the criterion for selecting tiie macro-block to 
embed the electronic watermark data 102 is described 
to be determined for a slice layer. However the criterion 
may be defined in connection with a picture, a field, a 
frame, or a GOP, in the scope of the invention. 

FurthernfK>re, it can be also understood that the 
embodiments of the invention can be easily imple- 
mented by way of a computer wrtii a program prepared 
in data-recording meda. 

Claims 

1 . Data-recording media wherein a program for imple- 
menting encoding of an original vkleo data by way 
of a computer into a compressed data stream hav- 
ing a multi-layer structure; said erxxxJing compris- 
ing steps of: 

storing a predetermined electronic watermark 
data in a main memory of the computer; 
transforming (82) the original video data into a 
sequence of processing units of frequency 
domain data, saki processing units being com- 
ponents of a predetermined layer of tiie multi- 
layer structure; 

emt)edding (S3) the predetermined watermark 
data into at least one unit of the sequence of 
processing units; and 

generating (86) ttte connpressed data stream 
by processing (84 and 85) the sequence of 
processing units. 

2, The data-recording mecfia recited in daim 1 ; saki 
encoding furttier conprising a step (T1) off deter- 
mining said at least one unit referring to a predeter- 
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mined aiterion. 

3. TTie data-recording media recited in daim 1; 
wherein: 

said encoding Is performed according to an s 
MPEG (Moving Picture Expart Group) standard; 
and 

said sequence of processing units is a 
sequence of macro-blocks d^ined in the MPEG 
standard. io 

4. The data-recording media recited in daim 2; 
wherein: 

said encoding is performed according to an 
MPEG standard; is 

said sequence of processing units is a 
sequence of nnacro-blocks defined in the MPEG 
standard; and 

said predetermined criterion is defined in 
connection with anyone of a slice, a picture, a field. 2o 
a frame, and a GOP (Group Of Picture) of the mufti- 
layer structura 

5. A vide&data eruxxJer for performing encoding of an 
original video data into a compressed data stream 25 
having a mufti-layer structure; said video-data 
encoder conrprtsing: 

means (101) for transfornung the original video 
data into a sequence of processing units of fre- 30 
quency domain data, said processing units 
being components of a predetermined layer of 
the multi-layer structure; 
means (103) for embedding predetermined 
watenmark data into at least one unit of the 35 
sequence of processing units; and 
means (104 ans 105) for generating the com- 
pressed data stream t>y processing the 
sequence of processing units. 

40 

6. The video-data encoder recited in daim 5; further 
comprising means (401) tor determining said at 
least one unit referring to a predetermined criterion. 

7. The video-data erKxxf er recited in daim 5; wherein: 45 

said encoding is performed according to an 
MPEG standard; and 

said sequence of processing units is a 
sequence of macro-blocks defined in the MPEG 
standard. so 

8. The video-data encoder recited in daim 6; wherein: 

said encoding ^ performed according to an 
MPEG standard; 

said sequence of processing units is a 55 
sequence of macro-bkx:ks defined in the MPEG 
standard; arxi 

said predetermined criterion is defined in 



connection with anyone of a slice, a picture, a field, 
a frame, and a GOP (Group Of Picture) of the multi- 
layer structure. 
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